This page describes the data flows for various platforms that produce data presented in the ODSS software system. This is a work-in-progress and expect changes to this as these evolve.
LRAUV data flow
There is a single script for both loading and conversion/interpolation from .nc4 to .nc called monitorLrauvHotSpot.py.
It should be put in a cron job that runs frequently enough to do the conversion as the files come in over hotspot.
Loads will happen sometime later when the .nc files are cataloged in the thredds server. Maybe once an hour ?
The monitor scripts uses the netCDF converter lrauvNc4ToNetcdf.py which
does the interpolation and resampling to create a CF-1.6 Discrete Sampling Geometry NetCDF file where all the variables
are interpolated onto the same time axis. This file can be loaded into STOQS. In this file raw data are interpolated
to 500 msec frequency then resampled at 2 second frequency.
Example syntax is as follows (on shore):
convertLrauvHotSpot.py -d 'CANON Spring 2014 ECOHAB experiment' -u 'http://elvis.shore.mbari.org/thredds/dodsC/LRAUV/daphne/realtime/hotspotlogs/' -o '/LRAUV/daphne/realtime/hotspotlogs/'
-b 'stoqs_ecohab2014' -c 'ECOHAB2014'
On the boat, you would replace the url with the threads server that is hosting the data there.
lrauv-hotspot-dataflow.graffle (application/octet-stream)
lrauv-hotspot-dataflow.graffle (application/octet-stream)
lrauv-hotspot-dataflow.graffle (application/octet-stream)
lrauv-hotspot-dataflow.png (image/png)
lrauv-hotspot-dataflow.png (image/png)
lrauv-hotspot-dataflow.png (image/png)
lrauv-hotspot-dataflow.png (image/png)
lrauv-hotspot-dataflow.png (image/png)
lrauv-hotspot-dataflow.png (image/png)
|
A shell script was added to the STOQS code base for running this script from cron: https://code.google.com/p/stoqs/source/browse/loaders/CANON/realtime/monitorLrauvHotSpot.sh And a cron job was set up on kraken to run it: 23 * * * * /opt/stoqshg/loaders/CANON/realtime/monitorLrauvHotSpot.sh > /dev/null 2>&1 
Posted by mccann at Apr 22, 2014 14:32
|
|
I like the idea of keeping the interpolation/resampling/binning code in python. This makes it easier to modify and/or maintain without digging into the compiled code (which is more tightly integrated into the executables that run on the vehicle).
Posted by mjstanway at Apr 24, 2014 11:50
|
